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Universal screening is one strategy to enhance the early identification of behavioral and emo- 
tional problems among youth. Although it appears to be effective, it is unclear if universal 
screening is more or less effective than current teacher referral practices. Thus, the purpose of 
this study was to compare the effectiveness of a teacher-rated, universal screener and typical 
teacher-referral methods in identifying youth at risk for emotional and behavioral problems. 

Results indicated that of the 24 students identified as at-risk by the universal screening mea- 
sure, only 1 1 were previously identified through current teacher referral practices - highlighting 
the potential benefit of universal screening to enhance early identification. Furthermore, results 
indicated that academic achievement and student engagement outcomes were significantly cor- 
related with at-risk status by identification method. The strengths and limitations of this study, 
as well as implications for practice, are discussed. 

A significant number of children are at-risk for or are currently experiencing emotional and behav- 
ioral problems (Ringel & Sturm, 2001; United States Public Health Service, 2000). The consequences 
for schools are clear such that emotional and behavioral problems have been well-documented to be 
significant barriers to learning (Catalano, Haggerty, Osterle, Fleming, & Hawkins, 2004). Children with 
an early onset of behavior problems are at elevated risk for academic failure, peer rejection, substance 
abuse, and delinquency (Reinke, Herman, Petras, & lalongo, 2008). Furthermore, national longitudinal 
studies show that more than half of the students identified with emotional or behavioral problems drop 
out of school, 75% achieve below expected grade levels in reading, and 97% achieve below expected 
grade levels in math (Bradley, Doolittle, & Bartolotta, 2008). Such findings overwhelmingly indicate 
that emotional and behavioral problems are associated with deleterious outcomes in youth. 

Despite a significant number of students experiencing emotional and behavioral problems, the ma- 
jority of these students remain unidentified and consequently untreated (Ringel & Sturm, 2001; United 
States Public Health Service, 2000). This is detrimental to student outcomes, considering that the longer 
a child’s emotional and behavioral problems go unidentified and untreated, the more stable his or her 
maladaptive trajectory is likely to be (Gottlieb, 1991). Early identification via screening is particularly 
important as it could help trigger early intervention, resulting in a disruption of the maladaptive trajec- 
tory. Research has also documented that recognition of a mental health problem increases the likelihood 
that children and their parents will seek help for that problem (Cauce, et al., 2002). In this way, universal 
screening efforts may ultimately lead to early intervention for students’ current problems as well as pre- 
vention of future problems. 

Within the school setting there is emerging evidence that early identification, combined with early 
and comprehensive prevention and intervention, can decrease the likelihood of academic failure and 
future life difficulties (Lane & Menzies, 2003; Walker & Shinn, 2002). Thus, as schools aim to serve all 
students regardless of risk level, through both special and general education supports, early identifica- 
tion via screening is a means for increasing the likelihood that more students are healthy, thriving, and 
progressing toward optimal development. Just as the fields of medicine and public health have repeat- 
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edly demonstrated the potential of early identification and intervention to prevent and alleviate disease 
and sickness (Fox, Flalpem, & Forsyth, 2008), the fields of psychology and education are beginning to 
demonstrate that using a similar approach within schools can do the same for children’s emotional and 
behavioral problems. Indeed, there is growing evidence that school-based screening can alter deleterious 
developmental trajectories and enhance positive outcomes (Report of the Alliance for School Mental 
Health, 2005). 

A key feature of early identification is a focus on assessments that are useful for identifying pro- 
gressive levels of risk among all students, not only among those with profound difficulties or problems 
(Glover & Albers, 2007). However, current methods of early identification vary widely, with many still 
focused solely on identifying students at the highest level of need. Such methods include teacher referral, 
parent referral for assessment and services at school or through primary care settings, youth self-referral, 
and universal screening. This study compares a common method for early identification-teacher referral 
to what may be a more underused and novel method universal screening. 

Traditional Teacher Referral 

Given that children spend countless hours at school, teachers are an invaluable resource for referring 
students in need of behavioral, emotional, and academic intervention. General education teachers are 
the primary link between students exhibiting problematic behavior and receiving access to school-based 
services. However, contemporary research indicates that teachers do not refer students based on behavior 
problems at the same rate as other academic concerns (Walker, Nishioka, Zeller, Severson, & Fell, 2000). 
In addition, many teachers perceive children’s emotional and behavioral problems as someone else’s re- 
sponsibility (Severson, Walker, Hope-Doolittle, Kratochwill, & Gresham, 2007) - possibly contributing 
to lower referral rates. Furthermore, previous studies have found a significant time lag between initial 
symptoms and referral to services by teachers (Duncan, Forness, & Hartsough, 1995). In general, such 
findings suggest that teaeher referral may not be the most effeetive method of early identification. Thus, 
the effeetiveness of alternative early identification methods, like universal screening, warrants further 
examination. 

Universal Screening 

School-based universal screening is conducted with all students in a given classroom, school, or 
district, to identify those at risk of aeademie failure and/or behavioral diffieulties and may therefore 
benefit from intervention (e.g., Severson & Walker, 2002). This approaeh toward early identification al- 
lows for the provision of evidence-based prevention and early intervention services delivered through a 
multi-tiered intervention approach (Kratochwill, Albers, & Shernoff, 2004). It is recommended that this 
multi-tiered approach be accomplished via a multi-gated screening strategy. However, what remains to 
be resolved is whether the information obtained at each gate adds significantly to the prediction equa- 
tion, justifying the additional time and resources required. In addition, key concerns regarding universal 
screening have been raised including the overidentification of students in need of school-based and com- 
munity mental health services, as well as the potential need for multi-informant assessments of youth’s 
behavioral problems (Levitt, Saka, Romanelli, & Hoagwood, 2007). Thus, further exploration is needed 
regarding the utility and feasibility of universal screening within schools. 

Current Study 

The purpose of this study was to evaluate the ability of a universal screening measure to identify 
students who might otherwise go undetected through a traditional identification paradigm (i.e., teacher 
referral for special education, child study team, or other school-based service provisions). Utilizing data 
from a cohort of third- and fourth-grade students, the differences between students referred and not re- 
ferred for evaluation or intervention based on the different referral systems was examined. Specifically, 
the aims of this study were twofold: 
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(1) Explore the differences between students identified as at-risk for behavioral and emotional dif- 
ficulties by a universal screening measure compared to those identified via teacher referral. 

(2) Examine the relationship of students’ academic achievement and engagement indicators to results 
from the early-identification method (i.e., universal screening or teacher referral outcomes). 

METHOD 


Participants 

Participants were 26 third-graders and 22 fourth-graders from two elementary schools in a suburban 
community, within the same school district, located on California’s central coast. During the 2008-2009 
school year, the total enrollment of one school was 286 students, and the total enrollment of the other was 
421 students. During that time, the demographic make up of both schools was comparable, with approxi- 
mately 73% of students identifying as Hispanic or Latino, 18% as White, and 9% as other or multiple 
ethnic groups. Approximately 68% of the students were classified as socioeconomically disadvantaged, 
40% as English language learners, and 14% as students with disabilities. Using class-wide data collec- 
tion procedures, the demographics of the participants in the present study (N = 48) were representative 
of the student population in these schools. 

Measures 

BASC-2 Behavioral and Emotional Screening System (BESS), Teacher Form. The BESS teacher 
form (child/adolescent version) is a screening instrument used to identify behavioral and emotional 
strengths and weaknesses of students in grades K-12 (Kamphaus & Reynolds, 2007). It assesses a wide 
range of behavioral problems and strengths, such as internalizing and externalizing problems, school 
problems, and adaptive skills. It is designed to be completed in 5 minutes or less. Respondents rate each 
item on a 4-point scale — never, sometimes, often, or almost always. The sum of the items generates a 
total T-score with high scores reflecting more problems. Scores of 20-60 suggests a “normal” level of 
risk; 61-70 suggests an “elevated” level of risk; and 71 or higher suggests an “extremely elevated” level 
of risk. The BESS was normed with a sample of 12,350 teacher, parents, and students, collected from 
233 cities in 40 states. Results from the norming process indicated that the psychometric properties of the 
BESS (across all forms) are generally acceptable, having good split-half reliability (.90-. 96), test-retest 
reliability (.80-. 91), inter-rater reliability (.71-. 83), sensitivity (.44-82), and specificity (.90-. 97). Fur- 
thermore, the measure has demonstrated acceptable convergent validity with the Achenbach System of 
Empirically Based Assessment (.71 -.77), Conner’s Rating Scales (.51-.78), Vineland Adaptive Behavior 
Scales (.32-. 69), Children’s Depression Inventory (.51), and the Revised Children’s Manifest Anxiety 
Scale (.55; Kamphaus & Reynolds, 2007). 

Report Cards. Student report cards include academic and student engagement indicators, as graded 
by their teachers. The academic indicators comprised 6 total subject areas — Listening, Reading, Writ- 
ing, Math, History, and Science — and corresponded to California state educational standards. Each indi- 
cator was graded on a scale of 1-4 (1 = has diffieulty with standard, 2 = approaehes standard, 3 = meets 
and applies standard, 4 = exceeds standard), indicating teachers’ perceptions of students’ present levels 
of achievement. For the purposes of this study, each subject area was conceptualized as a sub-composite, 
making up a total Academic Achievement composite. A behavioral engagement indicator accompanied 
each subject area, wherein the teachers graded the amount of “effort” students exhibited in meeting 
academic standards, using the same grading scale. Because these engagement indicators were unidi- 
mensional and few in number, they were summed into a total Engagement composite for the purposes 
of this study. 

Traditional Teacher Referral Data. Additional data was collected on each student to determine which 
students had previously been identified by teachers as being at-risk and needing additional behavioral or 
emotional evaluation or intervention. Noted indicators included: (a) referral to the school’s child study 
team, (b) testing for special education eligibility, (c) receipt of current special education services, and (d) 
receipt of other, non-special-education interventions (e.g., general-education counseling or classroom 
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environment alterations). These indicators were combined as dichotomous variables and students were 
classified as either “at-risk” or “normal” based on teacher referral for services. 

Procedures 

During the first quarter of the school year, the BESS teacher form was completed for all third-grade 
students attending one school and for all fourth-grade students attending the other school. For each grade, 
screening outcomes indicated that the majority of students were in the “normal” range, several students 
were in the “elevated” range, and relatively few students were in the “extremely elevated” range. Thus, 
for the purposes of this study, students receiving the latter two classifications were grouped together, 
resulting in dichotomized risk-level classification outcomes: “normal” (T scores 20-60) or “at-risk” (T 
scores 61 and above). Screening results indicated that 20 third-graders and 13 fourth-graders had BESS 
outcomes in the “at-risk” range. In an attempt to create matched groups, the 13 “at-risk” third-graders 
were selected to participate in the study, matched with a random selection of 13 “normal” third-graders. 
A random selection of 13 “at-risk” fourth-graders was then conducted, matched with a random selection 
of 13 “normal” fourth-graders. During the course of the study, however, 2 “at-risk” fourth-graders were 
transferred to another school, and so the matched pairs were reduced to 1 3 third-graders and 1 1 fourth- 
graders in each group (N = 48). 

After the sample participants were established, additional data was collected from school records 
and teacher interviews to establish which students were referred for additional services and previously 
identified by teachers as “at-risk.” Next, the students’ first quarter report cards - graded within a few 
weeks of BESS completion - were examined and coded. The Listening, Reading, Writing, Math, His- 
tory, and Science sub-composites were generated and weighted by summing the indicators associated 
with each subject area and then dividing that total by the respective number of indicators. The Academic 
Achievement and Student Engagement composites were derived via the same process as the sub-com- 
posites, using their respective indicators. 

Analyses 

All analyses were conducted using SPSS version 17.0. BESS scores and teacher referral for services 
were the two variables used to create the four proposed groups (see Table 1). Students were classified as 
“at-risk” via the BESS according to published norms for this measure (Kamphaus & Reynolds, 2007), 
and dichotomized for the purposes of this study. Students were identified as “at-risk” through teacher 
referral methods if they met at least one of four criterions: they were either (a) referred to the school child 
study team, (b) tested for special education eligibility, (c) currently receiving special education services, 
and/or (d) receiving other, non-special education intervention services. All analyses were conducted us- 
ing these aggregate grouping variables for BESS identification and teacher referral. 

Participant scores on the BESS and teacher referral were used to place individuals into one of the 
four groups. As shown in Table 1, Group 1 is labeled as Both Identified and consists of students classified 
as “at-risk” by both the BESS and teacher referral. Group 2 is labeled as BESS Identified and consists 
of students classified as “at-risk” by the BESS but not by teacher referral. Group 3 is labeled as Teacher 
Identified and consists of students identified as “at-risk” through teacher referral but not identified by 
the BESS. Group 4 is labeled as Not Identified and consists of students not identified as “at-risk” by the 
BESS or via teacher referral. 

RESULTS 

Results indicated that 23% percent of individuals were classified as Both Identified {n = 1 1); 27% of 
students identified as at-risk by the BESS (n = 13; BESS Identified) were not concurrently identified by 
their teachers as needing additional services; 8% of students were only identified as at-risk by teachers 
(n = 4; Teacher Identified) but not by the BESS; and the remaining 42% of students (n = 20; Not Identi- 
fied) represent a group that appears to be relatively free of behavioral and emotional concerns - not being 
identified via the BESS or teacher referral (see Table 1). 

A between-subjects multivariate analysis of variance (MANOVA) was conducted to investigate 
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the differences between groups on academic functioning and student engagement. Using the Wilks’ 
criterion, there was a significant effect for group membership, (F = 3.88, p < .001), indicating that aca- 
demic achievement and school engagement systematically differ among elementary students according 
to referral method. Results of follow-up analyses using Tukey’s tests are shown in Table 2, along with 
means and standard deviations for each group on academic achievement and student engagement vari- 
ables. Students in the Both Identified group had significantly lower academic achievement than students 
who were not identified as at-risk and those only identified as at-risk through teacher referral. Academic 
achievement mean differences were not significantly different among any of the remaining three groups: 
BESS Identified, Teacher Identified, and Not Identified. For indicators of student engagement, all students 
identified as at-risk by the BESS (in Both Identified and BESS Identified groups) had significantly lower 
scores than the Teacher Identified or Not Identified groups. 

TABLE 1: Student identification by referral method 


Teacher Referral 

BESS Teacher Rati nos 

At-Risk 

T>60 

Normal 

T<60 

At-Risk 

1. Both Identified 

III. Teacher Identified 

>1 referral 



Normal 

II. BESS Identified 

IV. Not Identified 

0 referrals 




Note. BESS indicates score from the Behavior and Emotional Screening System (Kamphaus c6 
Reynolds, 2007); Teacher referral indicates referral by general education teacher for additional 
assessment and/or services. 


TABLE 2: Mean Levels of Academic Achievement and Student Engagement by Method of Identifica- 

tion (N = 48) 


M ethod of Identification 


Both Identified BESS Identified Teacher Identified Noldentified 

(n=ll) (n=13) (n=4) (n=20) 


Dependent Variable 

M 

SD 

M 

SD 

M 

SD 

M 

SD 

Academic Achievement 

1.8U 

.44 

2.16=*’ 

.45 

2.41*’ 

.54 

2.56*’ 

.30 

Student Engagement 

10.36= 

2.20 

1— » 
1— > 
O 
O 

1.78 

14.50*’ 

1.73 

13.90*’ 

2.47 


Note, T ukey comparisons were empioyed to anaiyze group means in cases of significant F tests. Significant differences 
(p <.01) between group means are indicated by different letters. M eans having the same subscript are not significantly 
different. 


TABLE OF CONTENTS 


94 


The California School Psychologist, 2009, Vol. 14 


DISCUSSION 

These results provide initial evidence that universal screening may identify students not previously 
detected, or identified earlier than typically demonstrated, by current teacher referral practices. With 
an equal number of students identified as “at-risk” and “normal” by the BESS as part of this matched 
sample, it was anticipated that teacher referral practices would reflect these same rates. However, 13 
of the 24 students identified as at-risk by the BESS were not identified as at-risk by teachers. Thus, the 
BESS seems to have enhanced early identification potential - and possibly increased sensitivity - over 
teacher referral methods. Given that either the BESS or teacher referral (not both) could potentially serve 
as a first step in a multiple-gated early identification approach, it would be important that all students 
with potential emotional or behavioral risk are identified. While additional gates of assessment can fur- 
ther delineate which children are truly in need of services, children not identified via this initial gate are 
unlikely to receive the intervention services they need, and may continue untreated for a critical period 
of time before identification occurs. Given this conceptual framework, the results suggest that universal 
screening, when compared to traditional teacher referral methods, may serve as a more comprehensive 
method for a first-gate assessment. 

Significant mean differences were found between groups on measures of academic achievement and 
student engagement. Students identified as at-risk by the BESS and by teacher referral (the Both Identi- 
fied group) had significantly lower academic achievement scores than those not identified or those only 
identified by teacher referral. As the Both Identified group and BESS Identified group had statistically 
similar lower mean scores on academic achievement outcomes, this may further highlight the link be- 
tween behavioral problems and lower academic outcomes. Students identified by the BESS (regardless 
of teacher referral) had lower academic achievement scores, suggesting that a universal screener may 
identify students at-risk for emotional, behavioral, and academic problems. 

On measures of student engagement, two separate groups emerged. All students identified as at- 
risk by the BESS (either Both Identified or BESS Identified) had significantly lower student engagement 
scores than all students classified as “normal” on the BESS. The remaining Teacher Identified and Not 
Identified groups had significantly higher student engagement scores. These results indicate that teacher 
reports of student engagement levels may be an important consideration when identifying students who 
might be at-risk for future behavioral and emotional problems. Results obtained from a systemic screen- 
ing measure, aligned with pre-established school indicators (e.g., report cards), may assist teachers and 
school personnel in making well-informed data-based decisions. 

Strengths and Limitations 

Strengths of this study include the comparison of an already established method of student referral 
(i.e., teacher methods) to an understudied, relatively new method of student referral (universal screening 
via the BESS). This study demonstrated that universal screening potentially offers a more sensitive and 
efficient method of identifying students at-risk for behavioral, emotional, and academic problems. 

Limitations are present within the eurrent study. The generalizability of the findings may be lim- 
ited due to the small sample size, and the fact that both methods for comparison used the same rater 
(i.e., teacher). Further replication studies are needed with larger sample sizes and utilizing different 
rating forms (e.g., parent or self report) and methods (e.g., observations). Also, individual teacher-refer- 
ral methods may vary widely. Some teachers refer students for minor issues, while other teachers only 
refer students with severe emotional or behavioral problems. The BESS may be used as one element of 
a screening process to help make referral methods more consistent. 

Implications for Practice 

Results from the present study, alongside previous results investigating screening for emotional 
and behavioral risk, reveal that universal sereening may be a viable approaeh to early identifieation of 
students at-risk for behavioral, emotional, and aeademie problems. However, given limited finaneial 
resources, competing demands on time, and already high reporting requirements, many teachers and 
school districts may be skeptical of additional requests for student assessment. One approach, perhaps 
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more acceptable to school personnel, might be to integrate universal screening for emotional and behav- 
ioral problems into schools’ preexisting Response to Intervention (Rtl) paradigms. Data on emotional 
and behavioral problems could be collected alongside academic data to document which children are 
at-risk and could benefit from additional prevention or intervention. In this way, Rtl and multiple-gat- 
ing screening procedures could be integrated to improve the acceptability, precision, accuracy, and ef- 
ficiency of early identification, intervention, and prevention. Optimally, a continuum of school-based 
psychological services, starting with and grounded in universal screening to identify risk levels, could 
support students’ academic, emotional, and behavioral needs. However, as demonstrated by this study, 
universal screening may identify additional students not otherwise identified and may do so earlier than 
traditional methods. School personnel must be prepared to conduct further assessment and/or provide 
services for students in need. 
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